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ABSTRACT 

According to the survey of experts, the association of non-cognitive skills in groups 
according to their classification as object of evaluation leads to the fact that one group 
includes very different skills - both in volume and in the way of they identify and assess. 
So, the purpose of the research is the development of mathematical model of 
metathematic and metacognitive skills and abilities’ typology, as well as its subsequent 
extension to mathematical model of map of the present skills. As the basis for such 
typology codifiers developed in terms of FGES (Federal Governmental Educational 
Standard) requirements and corresponding systems of planned results within the 
framework of research on evaluation of education quality in elementary school, as well as 
within the framework of instrumentation for monitoring of dynamics of use in educational 
activity of “ICT as the instrument of activist approach realization” model by institutions 
of primary and general education and teachers of such educational structures was taken. 
The specified codifiers have been brought to unified codifier. Some objects of grading in 
unified codifier have been redefined in conformity with goals of the present research. 
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Introduction 

The survey of teachers on GlobalLab platform revealed the consolidation of 
noncognitive skills in groups by their reference to object of grading (the principle 
taken as a basis of codifiers) (Bolbakov, 2015) is perceived by teachers-practicians as 
artificial (Belova, 2014). In opinion of many respondents (Kazaeva, 2016), such an 
approach leads to the situation (Bulgakov, 2016) when one group includes 
absolutely different skills - whether by the volume or by the way of their 
distinguishing and evaluation (Bogdanov & Khorunzhiy, 2016). 

Materials and methods 
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Within the framework of the present research the more practically 
grounded approach is the development of n-dimensional space of skills with 
subsequent clusterization of skills by any of generally-accepted methods 
applying computer-assisted teaching (for example, the method of k-means) 
(Galiaskarova, 2015). Here we should underline the fact the applying of this 
classification method is being purely empiric and proceeds from the premise that 
such a method cannot possibly relinquish to codifier method, and even in the 
case of total failure in matters of distinguishing of explicit clusters will provide 
new data for research (Zaytseva, 2015). 

On the basis of above-mentioned principle we have developed 6 scales, each 
of which had been designed for characterization of definite noncognitive skill 
from the viewpoint of one of the following aspects: 

accessibility of observation by a teacher or a system of automatic 
journalizing; 

— temporal character of skill manifestation; 

— entirety of a skill in the context of its fixation; 

— “aspectuality” of a skill. 

Each of the six measurement scales possesses five discrete marks 
corresponding to the degree of manifestation of this or that attribute within a 
definite skill. Below there are list and explanations for all the developed 
measurements. 

Results, Discussion and Conclusions 

Graduality. The attribute of graduality reflects the ability of skill to 
manifest with various degree of intensity. A skill with high graduality degree is 
perceived as learning gradually, stage by stage. An observer dealing with such a 
skill can say it is “developed to a greater or a lesser degree”. Skills with low 
graduality degree can be described with binary opposition “there is a skill - no 
skill”. It is evident that graduality is closely related to the degree of heterogeny 
(diversity) of actions that define it. The most striking instances of skills with low 
graduality are often described by the only predicate. For example, reading, 
writing, talking, counting - all these skills are often perceived by an observer as 
binary. Nevertheless, it is evident that such a skill as reading or writing is not 
veritably binary. Thus, for example, in respect of reading we can speak about 
some degrees and stages of learning: knowing of letters, spelling, reading aloud 
and silently. However, in the majority of cases “the ability to read” is perceived 
as something that is either peculiar or not peculiar to an individual. In table 1 
there are explanations on scale marks for measurement of graduality. 


Table 1. Degrees of graduality attribute manifestation in respect of skills 


Mark on the 
scale 

Explanations 

0,2 

Skill is perceived as absolutely binary - it either is or not, there cannot be 
any interim options. An exaggerated example: the skill to add double-digit 
numbers. Such a skill can be easily and quickly checked by simple tests or 
fast observations. 

0,4 

Skill is perceived as close to binary. Observer can appraise its holding as 
incomplete. At that there are not many degrees of holding, they are 
strongly marked and follow one by one. Example: reading skill, in which 
four stages can be distinguished and all of them are clear-cut and short- 
termed (knowing of letters, spelling, reading aloud, silent reading). 
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0,6 

Skill is not perceived as binary, however its learning is being strongly 
marked. There is an easily-distinguishing stage when observer is ready to 
clearly confirm a pupil holds the skill. At the same time the observer quite 
easily distinguish cases when the pupil has learned the skill not to the full 
extent, partially or insufficiently. 

0,8 

Skill is perceived as gradual. Observer hardly distinguishes stages of skill 
formation. Stages of the skill are too vague, badly detached from each 
other, and the observer has doubts in their presence. In addition, the 
observer can make conclusions about presence of skill’s rudiments on the 
one hand, and can clearly distinguish high degree of its holding on the 
other hand. 

1 

Skill is extremely gradual. Observer often has no dead certainty at what 
stage of development the pupil’s skill is. Transitions between stages are 
very vague, their order is almost unevident. 

Stippling. The attribute stippling reflects temporal property of skill: in what 
interim it is observed, if its manifestation in time being extended or compact. A 
skill with high stippling degree is perceived as “quick”, “compact”, it can be 
distinguished within relatively short interim. A skill with low stippling degree 
manifests itself within large interim, which can often be explained by, for 
example, delayed character of result or large and extended in time variation of 
behavior corresponding to the skill. In table 2 there are explanations on scale 
marks for measurement of stippling. 

Table 2. Degrees of stippling attribute manifestation in respect of skills 

Mark on the r , 

scale Explanations 

0,2 

Skill is perceived as quickly manifesting in time. To distinguish it one 
needs just brief observation or testing of pupil. At that the skill can be 
rather complicated and include quite expanded variation of behavior. 

0,4 

Skill manifests itself in quite a fast way, however its diagnostics requires 
more than mere observation or rapid test. At that the first features of 
skill’s presence are distinguished rather quickly, but there is a necessity of 
additional observation for confirmation of initial impressions. 

0,6 

Skill possesses average stippling. Its quick distinguishing (due to 1 lesson or 
with help of 1 test) is rather difficult, however a teacher can put together 
those types of activity, which will promote faster manifestation of the 
skill. For example, research skills manifest much faster (during days or 
weeks) in the course of project activities, while in case of usual class- 
lesson system their distinguishing requires long terms. 

0,8 

Manifestation of skill is very extended in time. The variation of behavior 
linked to the skill consists of a big number of actions performed in long¬ 
term intervals. It is being very difficult to suggest a type of activity 
accelerating manifestation of the skill, however it remains possible under 
definite circumstances. 

1 

Skill is exceedingly extended in time (months, school year). It is often 
stipulated by high degree of its aggregating. Many teachers feel 
uncertainty in their ability to distinguish such a skill. Presence of such a 
skill is often stipulated with external global factors - such as effect of 
environment, family situation or even genetic predisposition. 


Granularity. The scale of granularity is designed for measuring degree of 
skill’s “separatability” against the background of other skills and their 
complexes. According to data of survey conducted on GlobalLab platform. 
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teachers intuitively subdivide skills into easily separated and hardly separated 
that are closely related to other skills or lost against their background. It is 
being particularly peculiar to the skills of regulative group. It’s very difficult to 
distinguish such skills, as far as the main objects of behaviour variations for 
them are often represented with other skills. Thus, for example, the skill of goal¬ 
setting is insomuch basic that almost always it appears shadowed by more 
particular actions and their objects. Moreover, it practically cannot be 
distinguished “in pure form”, since any test or task aimed at its distinguishing 
will obligatorily include a type of activity related to behavior variation 
associated with more granular skill - whether it is a communication, 
collaboration or self-regulation skill. In table 3 there are explanations on scale 
marks for measurement of granularity. 


Table 3. Degrees of granularity attribute manifestation in respect of skills 


Mark on the 
scale 

Explanations 

0,2 

Skill is being well-distinguishing under observations against the background 
of other skills. It possesses distinct features and is easily identified by 
several actions comprising corresponding variation of behavior. The very 
behavior variation consists of clear action that often possesses superficial 
expression. Observer experiences no difficulties in distinguishing and 
identification of the skill. 

0,4 

Skill is quite easily distinguished against the background of other skills, 
however observer can run into difficulties in recognition of set of observed 
actions sufficient for skill identification. Some actions of behavior 
variations have no clear superficial manifestation or are often met in other 
skills. 

0,6 

It’s rather difficult to distinguish skill against the background of other 
ones, however there are special conditions, upon which it can be 
identified unambiguously. It can also be easily distinguished in connection 
with other skills comprising more general aggregate skill. 

0,8 

Skill can hardly be distinguished as independent. The variation of 
behaviour linked to it possesses very weak superficial manifestation. The 
skill is not stipple, at the same time it is often gradual. 

1 

Skill is not distinguished as independent, which often makes many experts 
to draw a conclusion about its absence as independent. Its manifestation is 
very difficult to describe. Another one difficult task is the creation of tests 
or tasks that could diagnose it in pure form. It is often regarded as 
necessary condition for manifestation of some more granular skill. 

Testability. The scale of testability reflects the degree of how easily skill can 
be distinguished and identified with help of specifically developed test or task. 
In other words, when measuring testability one distinguishes the degree of 
observers’ confidence in fact the skill can be checked with help of automated or 
manual task. In table 4 there are explanations on scale marks for measurement 
of testability. 

Table 4. Degrees of testability attribute manifestation in respect of skills 

Mark on the 
scale 

Explanations 

0,2 

Skill can be tested. At that to check it one needs simple test that is often 
short and automated. Observers easily formulate external features of the 
skill and prepare tasks stipulating its manifestation. 

0,4 

Skill can be distinguished with help of test or task, which should be rather 
difficult. To make them one should apply definite efforts. The tests and 
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tasks are subject to partial automation, however checking requires “hand 
work”. 

0,6 

Skill can hardly be tested. Many observers agree that there is a necessity 
of such checking form as examination or interview to distinguish it. 
Preliminary diagnostics of the skill can use some partially automated tests. 
However, the skill is quite easily distinguished in case of rather long-term 
observation. 

0,8 

It is extremely difficult to distinguish skill even with help of examinational 
check or interview. It’s also very difficult to develop test or task, which 
could diagnose the skill even partially. Many observers agree that its 
identification requires long-term observation of pupil under condition of 
educational activity. 

1 

Skill cannot be tested. Its identification often requires very complicated 
abilities and competences on the part of observer. Many teachers rely on 
intuition in respect of diagnostics of such skills. To distinguish them there 
is a necessity of long-term observation of pupil including organization of 
various forms of educational activity. At that non-manifestation of 
features of skill presence even under long-term observation often does not 
imply absence of skill. The bigger part of actions comprising variations of 
behavior of untestable skills takes place without external manifestations. 

Observability. The scale of observability reflects general properties of 
superficial manifestation of this or that skill. Skills with low degree of 
observability possess extremely poor superficial manifestation, many elements 
of behavior variation possess no external manifestation as well and are being 
latent or remain invisible for observer. Skills with high degree of observability, 
on the contrary, possess good superficial manifestation, behavior variations 
corresponding to them manifest to the full extent in the form of sequence of 
clear-cut actions. In table 5 there are explanations on scale marks for 
measurement of observability. 

Table 5. Degrees of observability attribute manifestation in respect of skills 

Mark on the 
scale 

Explanations 

0,2 

Skill is well observed. Related variations of behavior include a big amount 
of actions with physical manifestation. Sequence of such actions is clear- 
cut and poorly intersect sequences related to other skills (the skill can 
likely be evaluated as possessing high granularity). 

0,4 

Observation of skill and its distinguishing are being difficult due to the fact 
the related variations of behavior include some number of actions, which 
possess no physical manifestation. However these “latent” actions are not 
of crucial significance for the skill. Often they are being secondary, 
facultative and not critical for achievement of result. 

0,6 

Skill is hardly distinguished by means of observation. Many actions 
comprising behavior variations related to it possess no external physical 
manifestation. At that some of these actions are critical for the skill. 
Despite the problems of observability such a skill can possess high or 
average degree of testability - there can be tests and tasks that diagnose 
it rather well. 

0,8 

It is quite difficult to distinguish skill by means of direct observation. 
Corresponding variations of behavior include a big number of actions 
possessing no external physical manifestation. It is also difficult to provide 
conditions for the situation when important elements of behavior 
variations manifest in the form of observed actions. 

1 

It is extremely difficult to distinguish skill by means of direct observation. 
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Often the conclusion of its presence is made on the basis of indirect 
indicators or achievement of successful result. Such skills often possess 
low degree of granularity and testability as well. In many cases observer 
relies on his experience and intuition in matters of discussion of these 
skills. 

Reflexivity. The scale of reflexivity reflects principal possibility of skill 
holder to distinguish the skill. The concept of reflexivity, as it is considered in 
the present research, is closely related to difference between skill and ability. In 
table 6 there are explanations on scale marks for measurement of reflexivity. 
Table 6. Degrees of reflexivity attribute manifestation in respect of skills 

Mark on the 
scale 

Explanations 

0,2 

Variation of behavior is easily described by skill holder. He can describe in 
details own actions that have led to positive result. Activity related to the 
skill can be attended by commentation. 

0,4 

Variation of behavior related to skill can be described by pupil. However 
such a describing often inexact, includes indications of actions that are 
not linked to the skill. Pupil hardly answers the question “what have you 
done for...” or “how did you manage to...”. 

0,6 

Variation of behavior related to skill is hardly realized by pupil. He 
experiences predicaments in describing own actions referred to the skill. 
Nevertheless, he can discern and describe some elements of behavior 
variation. 

0,8 

Pupil is almost not able to describe own actions led to successful result. 
He often describes own experience as the matter of course, something 
that do not require some control on his part. 

1 

Skill is not subject to reflexion to such extent that it’s often cannot be 
realized by pupils at all. It is extremely difficult to describe it in terms of 
variation of behavior. All the actions comprising the skill are perceived as 
sequence of operations performed automatically or in terms of reflexes. 


Falling back on the above-described conceptual model of complicated skills 
and abilities’ typology it becomes possible to formulate its heuristic 
mathematical model on the basis of linear-spline approximation method. 


Let us consider classical and the most wide-spread model for evaluation of 
the degree of holding knowledge, ability or skill based on ratio of number of 
successfully fulfilled tasks to general number of tasks. The model is represented 
in the formula (1): 


R = 



( 1 ) 


n 


where, R — mark; Ri — number of points obtained for execution of i-task (as a 
rule, the incorrect answer — 0 points, correct — 1 or more points); k — number of 
correctly executed tasks; n — general number of tasks (k <= n). 
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Afterwards the final mark is calculated by the following formula (2): 


1 ,R<c lt 

2 < R<c 2 , 


( 2 ) 


M,R> c 


M- 1. 


I — final mark, {cl, c2, ..., cM} — landmark value vector, M — maximum 
possible mark (for five-mark scale M = 5). 

Next we introduce the matrix 0 = {oik} assigning the degree of applying k-th 
complicated skill when executing i-th task (0 > Oik < 1). Besides we introduce the 
matrix V = {vik} assigning mark on 1-th scale described above, every k-th 
complicated skill (1 = {1,2 ... 6}). In such a way vector W for L of scales will assign 
weighted coefficients of every task, which will correct result from the viewpoint 
of application upon obtaining of various complicated skills and abilities. This 
vector can be taken by formula (3): 


W = Ox 


iy, 

i=i 

L 


( 3 ) 


The number of points, which is taken by tested as a result of execution of n 
tasks, can be defined by formula (4): 

n 

y = Tj w i x i’ {4) 

! = 1 


where xi — number of points taken by tested for execution of i-task. 

On the assumption of above-stated arguments the mathematical model of 
complicated skills and abilities’ typology is aimed at search of clusters {vik} and 
in such a way being a part of mathematical model of map of metathematic and 
metacognitive skills and abilities. Types correspond to clusters found with help 
of k-means++ algorithm in 6-dimensional space, which, in turn, form 
“landscapes” of the map 
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